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A time- varying empirical spectral process indexed by classes of functions is defined for locally 
stationary time series. We derive weak convergence in a function space, and prove a maximal ex- 
ponential inequality and a Glivenko-Cantelli-type convergence result. The results use conditions 
based on the metric entropy of the index class. In contrast to related earlier work, no Gaussian 
assumption is made. As applications, quasi-likelihood estimation, goodness-of-fit testing and in- 
ference under model misspecification are discussed. In an extended application, uniform rates of 
convergence are derived for local Whittle estimates of the parameter curves of locally stationary 
time series models. 
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1. Introduction 

In recent years, several methods have been derived for locally stationary time series 
models, that is, for models which can locally be approximated by stationary processes. 
Out of the large literature, we mention the work of Priestley (1965) on oscillatory pro- 
cesses, Dahlhaus (1997) on locally stationary processes, Neumann and von Sachs (1997) 
on wavelet estimation of evolutionary spectra, Nason, von Sachs and Kroisandt (2000) 
on a wavelet-based model of evolutionary spectra and more recent work such as Davis, 
Lee and Rodriguez- Yam (2005) on piecewise stationary processes, Fryzlewicz, Sapati- 
nas and Subba Rao (2006) on locally stationary volatility estimation and Sakiyama and 
Taniguchi (2004) on discriminant analysis for locally stationary processes. 

In this paper, we emphasize the relevance of the empirical spectral process for locally 
stationary time series. During the last decade, the theory of empirical processes has 
developed considerably and the number of statistical problems approached by utilizing 
concepts from empirical process theory is steeply increasing. In this paper, we show how 
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large parts of the existing methodology on empirical processes can fruitfully be used 
for time series analysis of locally stationary processes. In our set-up, the role of the 
empirical distribution of i.i.d. data is taken over by the empirical time-varying spectral 
measure. This generalizes a similar approach for stationary time series (cf. Dahlhaus 
(1988), Mikosch and Norvaisa (1997), Fay and Soulier (2001)). An overview of these 
methods and some references to the existing literature on empirical process techniques 
in other settings may be found in Dahlhaus and Polonik (2002). 

In Section 2, we introduce the empirical spectral process indexed by classes of functions, 
derive its convergence (including a functional central limit theorem) and prove a maximal 
exponential inequality and a Glivenko-CantcUi-type convergence result. These results use 
conditions based on the metric entropy of the index class. 

The empirical spectral process plays a key role in many statistical applications. In 
Section 3, we briefly discuss parametric quasi-likelihood estimation, nonparamctric quasi- 
likclihood estimation, inference under model misspecification by stationary models and 
local estimates. An extended application is given in Section 4, where uniform rates of 
convergence are derived for local Whittle estimates of the parameter curves of locally 
stationary time series models. 

Although our concept is based on empirical process techniques in the frequency domain, 
there exist many applications in the time domain. Section 3 and Section 4 contain many 
examples in the time domain, particularly with time-varying ARMA models. 

The empirical spectral process for locally stationary processes has also been briefly 
considered in Dahlhaus and Polonik (2006) in the special context of nonparamctric es- 
timation. In comparison to that paper, we also consider here the case of non-Gaussian 
processes and use weaker assumptions on the underlying process. We mention that the 
assumptions on the underlying process are very weak, allowing for jumps in the parameter 
curves by assuming bounded variation instead of continuity in the time direction. 

All proofs arc deferred without further reference to Section 5. 

2. The time- varying empirical spectral process 

In this section, we define the empirical spectral process and derive its properties including 
a functional central limit theorem and a maximal exponential inequality. 

2.1. Locally stationary processes 

Locally stationary processes were introduced in Dahlhaus (1997) by using a time- varying 
spectral representation. In contrast to this, in this paper, we use a time- varying MA(cxd)- 
representation and formulate the assumptions in the time domain. As in nonparamctric 
regression, we rescale the functions in time to the unit interval in order to achieve a 
meaningful asymptotic theory. The following assumptions on the locally stationary pro- 
cess are the same as those used in Dahlhaus and Polonik (2006). They are more general 
than, for example, in Dahlhaus (1997) since the parameter curves arc allowed to have 
jumps. 
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Let 

=sup|^|g(xfc) -5(a;fc_i)| : < xq < • • • < a;,„ < 1, m e N | (1) 
be the total variation of a function g on [0, 1] and for some k > 0, let 

IblW+^ljl, |j|>i. 

Assumption 2.1. Xt^n (^ = l,...,n) has a representation 

oo 

Xt,n= ^ at,nij)£t-] (2) 
j = -oo 

satisfying the following conditions: 

sup\at,n{j)\ < ^ (3) 
and there exist functions a(-,j) : (0, 1] ^ R with 

sup|a(u,j)| < (4) 



sup^ 



t=i 



at,„(j) -a( -,j 



n 



< K, (5) 



m-,j))<^. (6) 

r/ie £t are assumed to he independent and identically distributed with 'Est = and Ee^ = 
1. In addition, we assume that all moments of St exist and set K4 :=cum4(£t). 

Remark 2.2. (i) The rather complicated construction with different coefficients at,n{j) 
and a{^,j) is necessary since we need, on the one hand, a certain smoothness in the 
time direction (guaranteed by bounded variation of the functions a(u,j)) and on the 
other hand, a class which is rich enough to cover interesting examples. For instance. 
Proposition 2.4 implies that the process Xt^n = 4'{-)Xt-i,n + £t has a representation of 
the form (1). However, the proof of Proposition 2.4 reveals that this Xt^n does not have 
a representation of the form 

Xt,n= £ a(^,j)et-j. (7) 
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(ii) The time- varying MA(oo)-representation (2) can easily be transformed into a time- 
varying spectral representation as used, for example, in Dahlhaus (1997). If the et are 
assumed to be stationary, then there exists a Cramer representation 



V27t 7-7t 



cxp(iAt)d^(A), 



where ^(A) is a process with mean and orthonormal increments (cf. Brillinger (1981)). 
Let 



^i,n(A) ^ at,n{j)exp{-iXj). 



(8) 



Then 



1 



If (5) is replaced by the stronger condition 



cxp(iAt)At,„(A)dC(A). 



sup 
t 



at,nij) - a -J 
n 



< 



K 

n7(j)' 



then it follows that 



sup 



At,n{\)-A[ -,A 



(9) 



(10) 



which was assumed in Dahlhaus (1997). Conversely, if we start with (9) and (10), then 
the conditions of Assumption 2.1 can be derived from adequate smoothness conditions 
on A{u, A). 

Definition 2.3 (Time-varying spectral density and covariance). The function 

f{u,X):^l-\A{u,X)\' 

with 



A{u,X):^ X! a("'i)exp(-iAj) 

j = -OD 

is the time-varying spectral density and 



/(u, A) exp(iAfc) dA = a{u,k + j)a{u,j) 

-7T ■ 



J=-oo 

is the time-varying covariance of lag k at reseated time u. 



(11) 
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For a deeper understanding of the time-varying covariance, see also Proposition 5.4. 

A simple example of a process Xt ^ which fulfills the above assumptions is Xf ^ = 
4>{^)Yt, where Yt = a{j)et^j is stationary with |a(j)| < K/i{j) and (f) is of bounded 
variation. From the following proposition, it follows that time- varying ARMA (tvARMA) 
models whose coefficient functions are of bounded variation are locally stationary in the 
above sense. The result is proved in Appendix. 



Proposition 2.4 (tvARMA). Consider the system of difference equations 



' 'i\ ., A , /t\ ft-k 



E". y.^j..-., (12) 

j=0 ^ ^ fc=0 v / \ / 

where £t are i.i.d. with 'Eet = 0, Ejet | < oo, ao{u) = Po{u) = 1 and ctj{u) = aj (0) , l3k{u) = 
/3fc(0) for u <0. If all a.j{-) and as well as cr^(-), are of bounded variation and 

X]j=o 7^ for all u and all Q < \z\ < 1 + 5 for some 5 > Q, then there exists a 

solution of the form 

oo 

Xt^n = ^ at,n{i)£t-j 
3=0 

which fulfills (3)-(6) of Assumption 2.1. The time-varying spectral density is given by 

(u) lELo /3fc(»)exp(iAfc)p 



/(^^,A): 



27T lELo exp(iAj)|2 



2.2. Convergence of the empirical spectral process 



The empirical spectral process is defined by 

i?„(0) = xAI(F„(0)-F(0)), (13) 

where 

F{(p) = t r ^{u, \)f{u, A) dA du (14) 



J -n 



and 



Fni4>) = l^j2j" (15) 

with the pre-periodogram 

in'"^) ^ 27T ^ -'^[t+l/2+fe/2],n^[t+l/2-fc/2],nexp(-iAfc). (16) 

^ fe:l<[t+l/2±fc/2]<ri 
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If X[t+i/2+k/2],nX[t+i/2-k/2],n is regarded as a (raw) estimate of c(^,fc), then J„(^,A) 
can be regarded as a (raw) estimate of /(|-, A). However, in order to become consistent, 
J„(^,A) needs to be smoothed in time and frequency directions. The pre-periodogram 
Jn was first defined by Nemnann and von Sachs (1997). 

Many statistics occurring in the analysis of non-stationary time series can be written 
as functionals of F„(^). Several examples are discussed in Section 3 and Section 4. 

We first prove a central limit theorem for En{(t>) under the assumption that we have 
bounded variation in both components of 0(u, A). Besides the definition in (1), we need 
a definition in two dimensions. Let 

^ |0(uj,Afc) - (t){uj^i,\k) - 0(uj,Afc_i) +(/)(mj_i, Afc-i)| : 
< lio < ■ ■ • < < 1; -7T < Aq < • • • < Am < 7T; £, m e N > . 



For simplicity, we set 

\\(t>\\ooy :=supF(0(w, •)), ||0||y,oo supF((?!)(-, A)), 

ii A 

\\(j)\\v.v -^V^icj)) and ||0||oo,oo := sup |(^(u, A)|. 

ii, A 

Theorem 2.5. Suppose Assumption 2.1 is fulfilled and (pi , . . . , (f>k are functions with 
||0il|oo,V: ||0j||koo, ||0il|y,y and 00,00 being finite (j = l,...,fc). Then 

{En{(t>j))j=l,...,k (£'(0j))j = i,._fc, 

where (i?(0j))j=i,...,fc is a Gaussian random vector with mean and 
C0Y{E{4>^),E{4>k)) 

= 2n [ [ 0,(w,A)[(^fc(w,A) + </)fc(M,-A)]/2(w,A)dAdu 

Jo J -n 

+ ^4:J (^J 4>j {u, Xi) f {u,Xi)dXi^ J (f>k{u,X2)f{u,X2)dX2^du. 

Remark 2.6. (i) We mention that Theorem 5.3 contains a similar statement under a 
different set of conditions which is obtained as a by-product of our calculations. Further- 
more, we mention that Theorem 2.5 also holds if a data taper is used, that is, if Fn{(f>) 
and F{<f)) are defined as in (43) and (42) (in that case, we also need Assumption 5.1 
and COY {E {(f) j), E{(l)k)) must be replaced by c'"^\(j)j,4'k) as defined in Theorem 5.3). For 
simplicity, we consider the tapered case only in Section 5. 
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(ii) In contrast to earlier results (cf. Dahlhaus and Neumann (2001), Lemma 2.1), 
the assumptions on 0(m,A) and /(u,A) are very weak. In particular, we allow for non- 
continuous behavior. 

(iii) In the stationary case where (f>j{u, A) = (j)j{X) and /(w. A) = /(A), this is the clas- 
sical central limit theorem for the weighted periodogram (see Example 3.3 below). 

(iv) The limit behavior for complex- valued (jjj can easily be derived from Theorem 2.5 
by considering the real and imaginary parts separately. 

In Theorem 2.11, a hmctional central limit theorem indexed by function spaces and 
in Theorem 2.12 a Glivenko-Cantelli-type theorem are proved. The central ingredient of 
their proofs will be an exponential inequality for the empirical spectral process and a 
maximal inequality derived in the next subsection. 

2.3. A maximal exponential inequality 

Let 



P2{(f>) ■■ = 




and 

4.(0) := y^(K(<^) - EF„(0)). (18) 



Theorem 2.7 (Exponential inequality). Suppose Assumption 2.1 is fulfilled with 
Ejetl'" < for all G N. We then have, for all rj > 0, 

P{\Enm>v)<ciexp(^-C2^^-^'j (19) 

with some constants ci, C2 > independent of n. 

Remark 2.8. (i) In the Gaussian case, it is possible to omit the ^ in (19) or to prove 
a Bernstein- type inequality which is even stronger (cf. Dahlhaus and Polonik (2006), 
Theorem 4.1). 

(ii) The assumption Ejefj'^ < for all A: S N is strong in that it implies finite exponen- 
tial moments for St ■ On the other hand, with the above exponential inequality, this leads 
to a very strong finite-sample result for a large class of locally stationary processes (re- 
member that the assumptions made on the smoothness of the parameter curves are very 
weak). The strong assumptions on St make sense for obtaining the uniform bounds of the 
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empirical spectral process and the functional central limit theorem below. Furthermore, 
they will lead to a very strong uniform rate of convergence result in Theorem 4.1. 

(iii) To treat the bias Ei^„(0) - we set F+{(t>) '.^ ^^Eti I-n -/-(^^ A) /(i ^) dA. 

We then we have (see proof of Theorem 2.7 and Remark 2.8) 

V^|EF„(0) - F+(0)| < i^p2,„(0), (20) 

VH|i^+(<^) - Fiq^)\ < A (ll^ll^^^ + 11011^^^) (21) 

and 

P2A4>f < P2{^? + — !l<^|k,oo||0|U,oo, (22) 

n 

leading, for example, to the exponential inequality 

P{\E,,m>v)<c[exp\-c' "^^'^iM r^lM W^V (^3) 

I {P2[^) + (1/V^)|10|ly,oo + (l/V")ll'/'l|oo,oo V'^ J 

An alternative inequality used later is 

V^|EF„(0)-F+(<^)| < K^-^U\Uv + (24) 

The above exponential inequality is the core of the proof of the following result which 
then leads to stochastic equicontinuity of the empirical spectral process. Analogously 
to standard empirical process theory, stochastic equicontinuity is crucial for proving 
tightness. 

As for the standard empirical process, the results for the function-indexed empirical 
spectral process (£'„(0),(/) G $) are derived under conditions on the richness of <I>, mea- 
sured by the metric entropy. For each e > 0, the covering number of $ with respect to 
the norm p2 is defined by 

iV(e, $,P2) = inf{n > 1 : ...,(/)„€$ such that 

V0G $31 < i < n with P2{4>~ (j>i) < e} 
and the metric entropy of $ with respect to p2 By 

i/(e,$,P2)=logiV(e,$,P2). (25) 

Usually, the metric entropy is not known exactly, only upper bounds are known. These 
upper bounds are usually of the form i7(e,$,p2) < Ce~^ or A^(e,$,p2) < Ce^^ with 
C, r > 0. For the results below, we assume 

i7(e,$,P2)<i?*(e) 

with an upper bound i^$(-) which is assumed to be continuous and strictly decreasing. 
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Remark. In standard empirical process theory, so-called bracketing covering numbers 
are often used instead. Here, we do not use bracketing covering numbers since the em- 
pirical spectral process is not monotone in <j). 

Let 

Tooy ■= sup ||</'||oo,y, Tv,oo ■= sup ||(^||y,oo, 

ry^V := sup ||(/)||y,y and Too, oo := sup ||0|| 00,00 • 

In order to avoid further technical assumptions, the following results assume measura- 
bility of all random quantities without further mentioning it. 

Theorem 2.9 (Maximal inequality). Suppose Assumption 2.1 is fulfilled with 
E|et|'^ < for all /c G N. Suppose that $ is such that Too,v , tv,oo, tv,v and Too,c» 
are finite. Let 

T2 sup /92 ((/)). 

There then exists a set i?„ (independent of ^) with lim„_,oo ^'(Sri) = 1 and a constant 
L ( independent of $ and n ) such that for all rj satisfying 

77 ^ /Divmax-[Too,y,ryoo,T-yv, 



72 r ~ 



I H^isfds witha-.^H^^i'^ .—], (27) 
C2 Jo V 4 V '''2 / 

p(^xx^jEn{(t>)\>Tl,Br}j < 3ci cxp | - ^ (28) 



we have 



and 



p(^sup|K(0)|>r;,B„^ <3ciexp<|-^^/^;> (29) 



4 V -^2 



with some constants ci, C2 > independent of n. 



Remark 2.10. (i) A maximal inequality for {i?„(0), S <&}, assuming Gaussian innova- 
tions, can be found in Dahlhaus and Polonik (2006). The additional Gaussian assumption 
enables a weakening of the crucial assumption (27), essentially replacing H^{s) ds by 
Jq H^{s) ds. Related to that, the resulting exponential inequality is stronger, replacing 
in the exponent of (28) by ^ . The proof in the present non-Gaussian case is much 
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more complicated. It is an open question as to whether the same result as in the Gaussian 
case can also be obtained for non-Gaussian processes. 

(ii) The restriction to the set i?„ has several advantages. First, it allows for replacing 
P2,n{4>) by P2{4') (more precisely, by T2 = sup0g$p2(^)) which makes the results much 
simpler. Furthermore, extra terms due to the bias, as in (23), can be avoided. For many 
results, the set -B„ means no restriction, particularly if the probability of an event is calcu- 
lated as for equicontinuity. The set i?„ is given in (72) and it is shown in Lemma 5.9 that 
P(i3^) = 0{n~^). For this reason, may be removed from (28) and (29) by adding an 
0(ri^^) term to the right-hand side of these quantities. However, for many applications, 
this would not be sufficient. 

(iii) ci and C2 arc the constants from (19). The minimal choice of L is L = 
maxji^Ti, K2,K}, where Ki,K2 > are from Lemma 5.8 and K is the constant from 
(21) and (24). 

2.4. A functional central limit theorem and a GC-type result 

Theorems 2.5 and 2.9 arc the main ingredients for deriving the following weak convergence 
result for the process {i?„(0); G $} in the space £°°($) of uniformly bounded (real- 
valued) functions on $, that is, with ||5||$ := sup^g^ | (?((/)) |, we have £°°{^) ={g:^^ 
R; < 00}. 

Theorem 2.11 (Functional limit theorem). Suppose Assumption 2.1 is fulfilled with 
E|£t|'° < for all k G N. Furthermore let $ be such that Too,v, tv^oo> Tyy and Too, 00 
are finite. If in addition, 



as n ^ CO, where {E{(f)), £ $} denotes a tight, mean zero Gaussian process with co- 
variance structure as given in Theorem 2.5. 

Weak convergence in the above theorem means that 'Ei*a{En) 'E,a{E) as 71 —> 00 for 
every bounded, continuous real- valued function a on t°°{<^) equipped with the supremum 
norm, where E* denotes outer expectation. This Hoffman-j0rgensen-type formulation 
of weak convergence avoids measurability considerations for the process {iJ„ ((/>), <f> G 
Measurability of En might become problematic, particularly if <1> is not separable. 
Nevertheless, this notion of weak convergence allows the application of useful probabilistic 
tools such as continuous mapping theorems. For more details, we refer to van der Vaart 
and Wellner (1996). 

Finally, we mention our conjecture that it should be possible to prove another version 
of the above central limit theorem under much weaker moment assumptions on the 




(30) 



then we have 



En{-)^E{-) 



weakly in ioo{^) 
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if the class $ is smaller (by avoiding the use of the maximal inequality for proving 
equicontinuity) . 

As another application of the maximal inequality, we now prove a Glivenko-Cantelli- 
type theorem for the empirical spectral process. Here, we allow for a class $ = $„ which 
may be increasing with n. We set ^^P0g*„ l!</'lloo,y, etc. 

Theorem 2.12. Suppose Assumption 2.1 is fulfilled with E|et|'° < for all k G N. 

Suppose further that $„ is such that T^'y, Ty"^, '''{Pv '^"■'^ ''00,00 are of order o( jpg:i„ ); 
r^"' := sup^g^^ ^2(0) = o{y/n) and 

[ i^<E,„(s)2ds = o(^^). 
Jo 

Then 

sup |F„(0)-F(0)|= sup 



3. Applications 

In this section, we give several examples for the statistic Fn{(j))- In all cases, the results 
from Section 2 can be applied. As a non-trivial application, the uniform convergence of 
local Whittle estimates is proved in the next section. 

Example 3.1 (Parametric quasi-likelihood estimation). In Dahlhaus (2000), it 
has been shown that 

is an approximation to —log Gaussian likelihood of a locally stationary process. The 
above likelihood is a generalization of the Whittle likelihood (Whittle (1953)) to locally 
stationary processes. An example for a locally stationary process with finite-dimensional 
parameter 9 is the tvARMA process from Proposition 2.4 with coefficient functions being 
polynomials in time. Proving the asymptotic properties of 

9n := argmin£„(6') 

eee 

is greatly simplified by using the above properties of the empirical spectral process. We 
give a brief sketch. Let 
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be (up to a constant) the asymptotic Kullback-Leibler divergence between the true pro- 
cess and the fitted model (cf. Dahlhaus (1996), Theorem 3.4 ff) and 

^^0 argmin£(0) 

the best approximating parameter from Q (this is the true parameter if the model is 
correctly specified). We have 

£„(0) - m = -j=E„ (^fe') + ^iog(/e) 



with 



^ "^log/e^^jA^ - y log/e(u,A)du 



dA. (33) 



Thus, ignoring the i?iog-term, uniform convergence follows from the Glivenko-Cantelli- 
type Theorem 2.12. The i?iog-term can be treated as in Dahlhaus and Polonik (2006), 
Lemma A. 2. If Q is compact and the minimum 9o is unique, this implies consistency of 
^„.Lct V:=(J^,...,J^)'. Then 

VHV£„(0o)=-B„(^^V/g-i 
and 

The first term of V^£„(6') converges uniformly to 0, while the second term converges 
for 9 ^ 00 to the Fisher information matrix. The usual Taylor expansion then gives 
a central limit theorem for \/n{6n — do)- For details about the result and examples, 
we refer to Dahlhaus (2000), Theorem 3.1, the proof of which is greatly simplified by 
using the above arguments. Furthermore, the present assumptions are weaker. Due to 
(36) below, the result also covers the misspecified stationary case where the stationary 
Whittle likelihood is used with a stationary model but the true process is only locally 
stationary. 

Another application of the empirical spectral process is model selection for Whittle 
estimates. In Van Bellegcm and Dahlhaus (2006), a model selection criterion for semi- 
parametric model selection has been derived. Furthermore, an upper bound for the risk 
has been proven by using the exponential inequality for the empirical spectral process. 

Example 3.2 (Nonparametric quasi-likelihood estimation). In Dahlhaus and 
Polonik (2006), we have considered the corresponding nonparametric estimator 

/„ = argmin£„(5) 
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with 

where the contrast functional is now minimized over an 'infinite-dimensional' target class 
of spectral densities whose complexity is characterized by metric entropy conditions. 
The optimal rate of convergence has been derived for sieve estimates in the Gaussian 
case by using a 'peeling device' and 'chaining' together with an exponential inequality 
similar to the one in Theorem 2.7 (the exponential inequality is stronger due to the 
additional Gaussian assumption). It is an open problem as to whether the optimal rates 
of convergence can also be achieved for full nonparamctric maximum likelihood estimates 
or (as in the present paper) without the assumption of Gaussianity. 

Example 3.3 (Stationary processes/model mis specification by stationary mod- 
els). We start by showing how several classical results for the stationary case can be 
obtained from the results above. Let (t>{u, A) = (t>{X) be time invariant. Then 



Fn{<l>)= f m\Y.Jn{^K>^ dA. 



(35) 



However, we have 
. ft 




f t \ 11 

"^'^ I ^] ^ ?1 ^ 27t -'^[t+0.5+/c/2],ri-'^[t+0.5-fe/2],nCXp(-iAfc) 
^ ^ t=l l<[t+0.5+fc/2],[t+0.5-/c/2]<n 

(36) 



1 

2nn 



y^Xsexp(-iAs) 



/4A), 



where /„(A) is the classical periodogram. Therefore, f «(</>) is the classical spectral mean 
in the stationary case with the following applications: 

(i) 0(u, A) = 4>{X) ~ I[o,^J.\ {X) gives the empirical spectral measure; 

(ii) 0(u, A) = 4>{X) ~ 4^ V/g'^ (A) is the score function of the Whittle likelihood (similar 
to Example 3.1 above); 

(iii) (/)(u. A) = 4'{\) = cos Afc is the empirical covariance estimator of lag k. 

Theorem 2.5 gives, in all cases, the asymptotic distribution - both in the stationary case 
and in the misspecified case where the true underlying process is only locally stationary. 
In case (i) Theorem 2.11 leads to a functional central limit theorem on C[0,7t] with the 
supremum-norm. If ^(A) is a kernel, we obtain a kernel estimate of the spectral density 
(see the remark below). 
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Example 3-4 (Local estimates). There is an even larger variety of local estimates 
- some of them are listed below. The asymptotic distribution of these estimates is not 
covered by Theorem 2.5 since the function </>(«, A) depends on n in this case. However, 
in all cases, the uniform rate of convergence of these estimators may be derived by using 
the maximal inequality in Theorem 2.9. A detailed example is given in the next section 
where the uniform rate of convergence of local Whittle estimates is derived ((iii) below). 
For a short overview, let kn{x) = -^K{f-) be some kernel with bandwidth 6„. Then 

(i) 0(u,A) = k„{u — uo)fc„(A — Aq) gives an estimator of the time-varying spectral 
density /(mq, Ao); 

(ii) (j){u,X) = kniu — Uq) cosAfc gives a local estimator of the covariance function 
c(uo, fc); 

(iii) 0(it, A) = kn{u — Mo)|^V/g~"'^(A) is the score function of the local Whittle estima- 
tor of the parameter curve 6{ua). 

4. Uniform convergence of local Whittle estimates 

We now study kernel estimates for parameter curves of locally stationary processes and 
derive uniform consistency from the Glivenko-Cantelli-type Theorem 2.12 (see (39) be- 
low) and a uniform rate of convergence from the maximal inequality in Theorem 2.9. We 
investigate locally stationary processes where the time varying spectral density is of the 
form f{u, A) = .fe{u}W with 9{u) € 6 C R'' for all u e [0, 1]. An example is the tvARMA 
process from Proposition 2.4. 
Let 

Oniu) argmin£„(w, 0) 
see 

with 

We assume that the kernel K has compact support on [— ^, ^] and is of bounded variation 

with /^02 ^^(^) dx = and /j'^(/2 ^i^) dec = 1. Furthermore, let fe„ ^ and n&„ oo 
as n ^ oo . 

In case of a tvAR(p) process, 9n{u) is the solution of the local Yule- Walker equa- 
tions: Let c„(w,fc) := i^^t ■i:K{^^-^)X[t+i/2+k/2],nX[t+i/2-k/2],n (cf. Proposition 5.4), 
C„{u) = {cn{u, 1), . . .,c„(w,p))' and i;„(u) = {c„(u, i - j)}i j=i,...,p. If 6'„(u) = (ai(u), . . . , 
ap(w), (T^(u))', then it is not difficult to show that 

(ai(u), . . . , ap{u)y = -E„(m)"^C„(u) 
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and 



p 



(u) = c„ (w, 0) + ^ Sfc iu) Cn{u,k). 

k=l 



We now derive a uniform rate of convergence for 0„(u). Let V:=(g|^, . . . , , II ■ II2 be 
the ^2-norm and ||A||spec '■— sup^-gcn 1^12^ spectral norm (where A is an n x n 

matrix) . 



Theorem 4.1. Suppose Assumption 2.1 is fulfilled with E|et|''' < for all k £N and 

time-varying spectral density f{u,X) = feQ{u}W- Suppose, further, 

(i) 9 is identifiable from fg (i.e., fe{X) ~ for all A implies 9 ~ 9' ) and 9o{u) 
lies in the interior of the compact parameter space & C R'' for all u; 

(ii) 9o{-) is differentiahle with Lipschitz continuous derivative; 

(iii) /0(A) is twice differentiahle in 9; fg^{\) and the components of V fg{\) and 
W'^ fg{X) are uniformly hounded in A and 9 and uniformly Lipschitz continuous in A; 

(iv) the minimal eigenvalue of I (9) := J_^(Vlog/e(A))(Vlog/9(A))' dA is bounded 
away from uniformly in 9. 

We then have, for bnU ^ (logn)^, 



Remark 4 •2- (i) We conjecture that a similar result also holds in case of model mis- 
specification where the model spectral density fgg(u)W is only an approximation to the 
true spectral density /(it, A). 

(ii) It is possible to extend the above result to a wider range of smoothness classes as, 
for example, in Moulines, Priouret and Roueff (2005). This is not too difficult since only 
the estimation of the second summand of (41) below needs to be improved in an obvious 
way (and the kernel K needs to be replaced by a higher order kernel). 

(iii) For tvAR(p) processes, it follows from Theorem 4 in Moulines. Priouret and RoucflF 
(2005) that the above rate is the optimal rate of convergence. 

(iv) Of course, the assumption E|e(|'^ < for all fc G N is restrictive in comparison 
with what one would expect for standard estimation results based on spectral analysis. 
It is the price that must be paid for a uniform convergence result with optimal rate and 
an elegant proof with the maximal exponential inequality. 



sup 

ttG [b„/2 , l-fc„/2] 




that is, for b„ ^ n 



-1/5 



, we obtain the uniform rate Op{n ^/^). 



Proof of Theorem 4.1. We begin by noting that the difficult parts of the following 
proof are handled by using the empirical spectral process and applying Theorems 2.9 
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and 2.12. We start by proving consistency. We have, with 

:.i;£|,<.g4„"/.(A) + &i>}dA, 

W,-C(uS = -Ls.(i-A'(!i-) 

4nJ_„Jo bn V bn J /e(A) 

1 1 T^f^^ i/"- 



We now apply Theorem 2.12 with = ® ^/"^ \u G [6„/2, 1 - bn/2],0e 

&}. It is straightforward to show that N{e,^n,P2) < K / ib'h^^^^'^e'^+^), that is 

/o'i?*„(s)2ds = 0((log &„)')• 

Furthermore, , r^"^ , Ty"y and t^^oo arc of order 0{b~^) and Ta"^ = sup^g^,^ p2 (0) = 
0(fori^^^)- Thus, for bnU > fog^n. Theorem 2.12 imphes that 

sup sup \Cniu,e)~c{u,e)\^o. (38) 

[fc„/2, i-6„/2] eee 

The identifiabihty condition imphes that 9o{u) is the unique minimum of C{u,9) for aU 
u. By using standard arguments, we can therefore conclude that 

sup \\en{u)^eoiu)\\2^o. (39) 

m6 [6„/2 , l-6„/2] 

We now derive the rate of convergence by using the maximal inequality. We have, for 
each u, 

Zn{u) :=VC„{uX{u)) -VCn{u,eoiu)) =V^Cn{uJ„iu))(0n{u) - Ooiu)) (40) 

with \9n{u) — Oo{u)\ < \9n{u) — 9o{u)\. The main term on the left-hand side is 
d 

—£„(■(/, 6*0 (u)) 

= -^EM + I" ^ 1k(^^^ {feo(v)W - .feo(n)(A))^/,-\A)|,.,„(„)d«dA 



with (/)„(u,A) := ^A'(i^)^^/g {X)\e=0„(u)- We apply the maximal inequality of 



Theorem 2.9 to the class = {r-A'(T^) ® T^M-ZfT^ I ^ [6„/2, l-6„/2], 6* € 6}. Again, 
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we can show N{e, $„, P2) ^ K / (6l''+^^/^e''+2), that is, i/$„ {sf As = 0{{\oghnf). Fur- 
thermore, r^V' '''vvLi '''v}/ ^'^^ T-i^^oo are of order 0{h~^) and Tj"' = sup^g^^^ P2(0) ~ 
6„ We now apply Theorem 2.9 with rj = T2S for arbitrary d. If &„n ^ log® n, then the 
conditions (26) and (27) are fulfilled and we obtain 



P(^sup \En{(f>)\>T2S, B,,^ <3ciexp|-^%/J 



and, as a consequence, 



sup 



The smoothness conditions (ii) and (iii) imply that the second summand of (41) can be 
uniformly bounded by 0(6^), that is, we obtain 



sup ||V/:„(m,6'o(u))||2 = Op 

ti6[6„/2,l-b,i/2] 



1 



n — 



If 9n(u) lies on the boundary of 8 for some u, then ||6'„(m) — 6'(?i)||2 > k, for some k > 
and, by (39), 



P 



sup ||V£„(w,^„(m))|12 > (5-^ 

tiG [6„/2,l-6„/2] VOn?^ 



sup ||V£„(ii,6'„(u))||2 >0 

tie [6„/2,l-fc„/2] 



sup ||6I„(w)-6Io(w)|12>k) ^0, 

ue [t„/2,l-b„/2] 



implying that sup^^g [(,^/2.i-&„/2] ll-Z^ra('«)||2 = ^pi ^^ „ ) ■ order to obtain the assertion 
of the theorem from (40), we now prove that the minimal eigenvalue of the matrix 
V^>C„(w, 0„(w)) is bounded away from uniformly in 6 in probability. Wc have 



V On 



1 ^Ju- 



An 



1 ^ 1 fu-t/n\ 1 r 



47T 



(Vlog/e(A))(Vlog/e(A))'dA. 



Since /e is twice differentiable in 6 with Lipschitz continuous second derivative in A we 
obtain, exactly as above from Theorem 2.12 for 5„7i > log^n and i,j = 1, . . . , d, 



sup sup 

ue[6„/2,i-h„/2]eee 



1 



1 



E„, —K 



1 ,-1 
Je 



47T (961, a6lj 
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Therefore, also, 



sup 

ti6[b„/2,l-b„/2] 



Er, —K 



0. 



spec 



Since the minimal eigenvalue of I{9) ^ J'"^(Vlog/6i(A))(Vlog/g(A))'dA is bounded 
from below by Ainin(^) > uniformly in 9, this implies that 



Since 



sup ||V^£„(u,6'„(u)) ^|jspec<T TTT 

tiG [f)„/2,l-f)„/2] Amin^-'j 



3n{u)-Bo{u)\\2<\\V^Cn{u,9n{.u))-^\Up,,\\Zn{u)\\ 



this implies the result. 



□ 



5. Proofs: CLT and exponential inequality 

In this section, we provide the proofs for the results of Section 2. In particular, we derive 
the asymptotic behavior of the moments of the empirical spectral process. 

First, we extend the definitions of Section 2 to tapered data X^"'^ = hn{-^) ■ Xt^n, 
where /in : (0, 1] [0,oo) is a data taper (with hn{-) — I(o,i]{-) being the non-tapered 
case of Section 2). This is done for three reasons: 

(i) The main reason is that all proofs arc greatly simplified since the data taper now 
automatically takes care of the range of summation (h„ {t/n) is zero for all t outside the 
observation domain {1, . . . ,n}). The consideration of arbitrary tapers /i„ instead of the 
'no-taper' /(o,i] docs not introduce any extra technical complexity at all. 

(ii) By using tapers which are different from only on a segment of the observation 
domain, one may construct localized estimators. This is of great importance, although 
wc will not discuss it in the present paper. 

(iii) The use of a data taper in the periodogram is standard for stationary time series. 
It leads to a better small-sample performance in the presence of strong peaks in the 
spectrum. It may turn out that this also holds in the present situation (which requires 
further investigation). 

As before, the empirical spectral process is defined by Eil'"\<j)) := ^^{Fi^-\(j,) - 
F^^^\(t>)), where 

F^^-\4^) [\l{u) r ^{u,\)f{u,\)d\du (42) 

Jo J -n 



and 



i^(^")(0) 



(43) 
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now with the tapered pre-periodogram 

^ ^ k:l<[t+l/2±k/2]<n 

We mention that in some cases, a rescahng may be necessary for Jn'^"\u, X) to become 
a (pre-) estimate of /(u,A). 

Throughout this appendix the superscript (hn) will be dropped in many 
situations for notational convenience, that is, we will use Fn{4>), F{cj)) and 

Assumption 5.1. The data taper /i„ : (0, 1] [0,oo) fulfills sup„l/(/i„) < C and 
supjj „ hn(u) < C for some C < oo. Furthermore, log/i„(-) is concave. 

The assumption that log/i„(-) is concave is very mild (note that even log(a;™) is con- 
cave). We need the following notation. With 

/7T 
(/)(u,A)exp(iAj)dA, (45) 

we define 

oo oo 

Poo(0):= J2 sup|0(w,j)l and i;e(0) := ^ V{^{;j)). (46) 

u 

J — — 00 J — — 00 

We mention that 

sup|(/)(u, A)| < ^/9oo(<^) and p2(0) < ^= Poo(0)- 

The idea now is to prove the CLT in Theorem 2.5 by the convergence of all cumulants. The 
convergence of the cumulants is derived below under the assumptions Poo(</>) < oo and 
V'si4') < Unfortunately, these assumptions are not fulfilled for functions of bounded 
variation as assumed in Theorem 2.5. Therefore, its proof also uses certain approximation 
arguments (cf. proof of Theorem 2.5). The following CLT is a by-product which follows 
immediately from the cumulant calculations below. It is of independent interest since 
the result does not follow from Theorem 2.5 (the condition poo(0) < oo does not imply 
bounded variation in the A-direction; furthermore, the conditions may be easier to check 
in some situations). 

Assumption 5.2. Suppose (f>: [0,1] x [— tt, ttJ^-R fulfills p^a {4') < and vy,{4)) < oo . 

Theorem 5.3. Suppose Assumptions 2.1, 5.1 and 5.2 hold with a data taper h indepen- 
dent of n. Then 
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where {E'-''\(j)j))j=i^„ k is a Gaussian random vector with mean and coM{E^^\(j)j), 



E(^){ct)k))=c^E\<Pj,c^k) with 



4^^(0j,0fc):=27T / h\u) I 4>,{u,\)[<t>k{u,\) + <t>k{u,-\)]f{u,\)d\du 



— 7T 



+ AC4y h {u)yj Ai)/(u, Ai)dAi j A2)/(u, A2) dA2j dw. 

Proof. The result follows from the convergence of all cumulants which is proved in 
Lemma 5.5(ii), Lemma 5.6(ii) and Lemma 5.7(iii). □ 

In the same way, one may arrive at a central limit theorem with (||/in||2^"'^£'n(0j))j=i,....fc 
for hn dependent on n (mider additional assumptions), for example, for /i„(-) = 
/[„(,_t,^/2.Mo_b„/2] (■) (segment estimate). This will be studied in future work. 

For the following proofs, we first need a result on the behavior (decay) of the covariances 
of the process. The case hn{-) ~ /(o,i](') gives the results for the ordinary covariances. 

Proposition 5.4. Suppose Assumptions 2.1 and 5.1 hold. We then have, for all 
k, ki, k2 e Z with some K independent of k, ki,k2 and n, 

sup|cov(4t"\<I.!„)l^^' (47) 
sup|c(«,fc)| <^, (48) 



E 



m,^))<|y. (50) 



Proof. From (11) and (4), we have 

CO 

c{u,k)^ E a{u,j + k)a{u,j) with sup |a(u,j)| < ——. 

j = -oo 

Therefore, (48) follows from the relation 
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which is easily established. Furthermore, we have, with k = ki + k2, 

^ ^ ^ ^ J — — oo 

For ki =k and ^2 =0, this gives (47) by using (3). Replacing /i„(^±^), h„{*-^), 
at+kun{j + k) and at^k2,nU) by ^n(^), ^n(^); o,{^,3 + k) and a(^, j), respectively, gives 
(49). For example, the last replacement step has, with (5) and (6), the upper bound 




Since \k2\ < |fc| + |fci|, we obtain (49) for |fci| < n. For |fci| > ri, the result follows since 
then cov(Xj^^| n^-^t^ko n) equal to 0. (50) is obtained in the same way. □ 

A trick which greatly simplifies the following proofs is to set at^nU) = for i ^ {1, . . . , 71} 
and J G Z, a{u, j) = for (0,1] and j € Z, (j>{u, A) = for u ^ (0, 1] and A e [-7t, 7t] 
and hn{u) = for u ^ (0, 1]. With this convention, (3)-(6), (47)-(50) continue to hold for 
u S R, i G Z, V{f) now denoting the total variation over R and t in (5) and (49) ranging 
from —00 to 00. Furthermore, the summation range of k in (16) and t in (15) can be 
extended from —00 to 00. Therefore, all summation ranges are from —00 to 00 in the 
following proofs unless otherwise indicated. 

We also set a{j) = sup„ \a{u,j)\, <j){j) = max{sup„ |0(m, j)|,sup„ |0(m, -j)|} and c{j) = 
sup„ \c{u,j)\. 

Lemma 5.5. (i) Suppose Assumptions 2.1 and 5.1 hold and (f) : [0, 1] x [— tt, tt] -^H is a 
function possibly depending on n. We then have, with K > 0, 

\k\<n \k\>n ^ ' k ^ ' 

ill) If, in addition, (j) is independent of n and fulfills Assumption 5.2, then 

E4'^")(0) = O(n-i/2). 
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Proof, (i) We have 



2nn 

t=i k 

We therefore obtain from Proposition 5.4 



(52) 



t, |fc|<n 



n) 



27m ^ " I " 



t,fc 



with 



rj ^ — ^ 



|fc|<n 



n / \ n 



min(|/c|, 7i) 



-,-k]c[ -,k]+R 



(53) 



£(fc) 



^E^(^)^^^po.(^)- 

|fc|>n ^ ' 



(54) 



Furthermore, 



k 

'tt ^ 



n / \n 



t,fe 



i/?i 



n \n 



k c 



t~l 



+ 2- U 



t-1 



+ X, —k I c 



1 



+ X, A: 



dx 



(55) 



leading to the result, (ii) follows immediately. 



□ 



Lemma 5.6. (i) Suppose Assumptions 2.1 and 5.1 hold and (f)i^(f>2'- [0, 1] x [— tt, tt] R 

are functions possibly depending on n. We then have 

COv(£;(''")(0i),£;(''")(02)) = 4^"^(<^l>2) + Rn 



with 



K 



\Rn\ < - E Mki)h{k2 

71 ^ ^ 



ki .k-2 

K 



min{|fci|, n} 
^(fci + fcz) 



+ - [Mki)v{M-M)V{M-M))h{k2)] 



ki .k2,k2 



min{|fci| + |fc2| + |fc3|,n} 
e{k3)e{ki +k2 + fca) 
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K 



+ - E iMki)v{M-,k2)) + V{4>i{-M))h{k2) 

n ^ — ^ 



ki ,^2 



where the last term can be omitted if Xt.n is Gaussian. 

(ii) // 01 and 02 are independent of n and fulfill Assumption 5.2, then _R„ = o(l). 



Proof, (i) Wc have, with (52), 

cov(i?(''")(0i),4''"H02)) 

1 



(27t)2n 



E 



X COv(Xj'j^;^\y2-fci/2],n'^[t2 + l/2-fe2/2],ri) 
X C0v(Xj'-j^^\^2-fci /2] ,n^[t2 + l/2+fe2 /2] ,«) 



(/In 



cum(X| 



(/In) 



X, 



[ti + l/2+fci/2],n' [ti + l/2-fei/2],n' 



X, 



[t2+l/2+fe2/2],n' [t2 + l/2-fe2/2] 



J]- 



(56) 



Let fca := ii — i2 + [fci/2 + 1/2] — [fc2/2 + 1/2]. By using Proposition 5.4, we replace the first 
summand in [. . .] by /i„(— )^c( — , A;3)c(— , fcs + ^2 — fci). The remainder can be bounded 
by 



K 



n 



E 



^1,^2,^3 



0l(fcl)02(fc2) 1 + 



min{|fci|, n} 



■0l(^l)02(fc2)- 



1 



i{kz) jiik3 + k2-ki) 
^ ^ min{|fci|,n} 



^(fc3)l ^(fc3 + fc2-fci), 

(51) imphes that this is bounded as asserted. Therefore, the first term is equal to 

f ti + ko 



02 



ti ,ki ,^2 ,^3 



,^1 M J 

X C| — , fc3 C — , fc: 
n J \ n 



\ n 
3-r ki] + Rn, 



(57) 
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where h = -fcg + [fci/2 + 1/2] - [k2/2 + 1/2]. Replacing ^.2{ii±I^,~k2) by 02(^,-fc2) 
yields the error term 



K 



n 



which is also bounded as claimed. As in (55), we now replace the i sum in (58) (with 
ko = 0) by the integral over [0, 1] with the same replacement error. Direct calculation (or 
repeated application of Parseval's equality) yields 

„ / h^i'u) ^ (j>i{u,~ki)(j>2iu,~k2)c{u,k3)c{u,k3 + k2-ki)du 

n 1 pTX. 

= 27t/ hi{u) (j)i{u,\)(t)2{u,-\)f{u,\YA\Au. 

Jo J-n 

The second term in (56) is treated in the same way. With the representation Xt.n = 

YjT=~oo°-t.n{i - j)£j and the abbreviations t+ = t^{t^,k^) [t^ + 1/2 + k„/2], = 
t~{ti,,ky) = [iu + 1/2 — fc^/2], the third term is equal to 

77^h~ yi ^i( — ) <^2( — 



^.„(i)/.„(f).„(^)^.(f) 



By using Assumption 2.1 and (51), we now replace this by 



(n \2 5Z 0l( — ,-^1 I 02( — 



a(|,t^^.) (58) 



, <1 _ \ /<2 + \ { t2 - 

n J \ n J \ n 



,t.2 -i) 



with replacement error /vri ^ ^^.^ ^.^ (f>i{ki)(j>2{k2)- We now replace the term a(-^ 
in the above expression by a{^,t2 — i), leading, with the substitutions d ^ t2 — ti and 
J = i — ti; to a replacement error of 

- ^ <^;(fcl)02(fc2) 

ki , k2 
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^ 0(\l. _i_ * 



E 



2 2 



2 2 



The last sum is bounded by |(i|y(a(-, [d + i — — j)), leading to the upper bound 



K 



- V <^~i(fci)02(fc2) 



xE 



\[d+h + ¥]-j\ + \[d+h-^]-j\ + \[h + ^]-j\- 



2 2 J 



n ^ — ^ 



for the replacement error. In the same way, we replace (59) by 



K4 



(27t)2 n 



*1 



X a| — , 

n 



i ) a ( — , t J — j ) a ( — , <2 



(59) 



K4 



(27T)271 ■ 



,-fc2 ) c( — ) c( — 



with replacement error Kn ^ X]fci,fc2 '^~i('^i)['^2(fc2) + V^((^2(-, fe))] [j^^ + j^l^j]. As in 
(55), we now replace the i sum by the integral over [0, 1]. Application of Parseval's 
equality gives the final form of the fourth order cumulant term. 

(ii) Considering the cases |fc| < ^/n. and > \/n separately shows that 



i Vmin{|fc|,n}0j(fc) =0(1) and - V min{|A:|, n}— ^ = o(l). 
n ^ n ^ l(k) 

This implies that the first term of _R„ tends to zero. Since 
min{|fci| + |fc2| + |fc3|,n} 

< 2 min{ | fci + A;2 + fca | , nj + 2 min{ | fci | , n} + 2 min{ | feg | , n} 

and \V , k))\ < K, the third term of i?„ also tends to zero. 



(60) 



□ 
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We now set 

1/2 



P2,n 



Note that p'2n{^) ~ P2,n{4') in ttic non-tapered case where hn{-) ~ ^(o,i](')- 

Lemma 5.7. Suppose Assumptions 2.1 and 5.1 hold and 0i, . . . , </)f : [0, 1] x [— tt, tt] — > R 

are functions possibly depending on n. 

(i) Ifi>2, then 

I 

with a constant K independent of n. 

(ii) If ( >2 and, in addition, E|£t|'^ < for all k ^ N for the £t from Assump- 
tion 2.1, then 

|cum(4'-)(0i), . • . < K\2i)\{{p^l-\cj,,) (62) 

with a constant K independent of n and t. 

(iii) If i>-i and , . . . , 0^ are independent of n and fulfill Assumption 5. 2, then 

|cum(i?i''")(0i), • • . = 0{n'-'/'). 

Proof, (i) We have, with (52), 

cnm{Ei!^-\cP,),...,Ei'-\<t>i)) 



= n^/2cum(F,^)(<^i),...,Ff")(0, 



(27T)V/2 ^ ^ ^Mti 



''[ti-|-l/2+A:i/2],n'^^ [ti -|-l/2-fci /2] ,n ' 
• ■ • ^ ^[tt + l/2+ke/2],n^[ti + l/2-ke/2],n)- 

We now use the representation Xt^n = '^^L-oo ^t,nit — j) £j and obtain, with the product 
theorem for cumulants (cf. Brilhnger (1981), Theorem 2.3.2) and the abbreviations t+ = 
t+{t^, k^) = [t^ + 1/2 + fc^/2], t- = t-{t^, k^) = [t^ + 1/2 - fc^/2], that this is equal to 

1 

(27t)^n^/2 
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E E ' 

ti,...,ti ki,...,ki 



ti 



X E n 

ti,...,ie,ji....,ji v=l 



te 



/in Un (t+ -i^)a^^ (t^ -j^) 



X J|cum(e,|s e Pj): 

{Pi,...,p,„}i.p.j=i 

where the last sum is over all indecomposable partitions (i.p.) {Pi, . . . , Pm} of the table 

ii ji 

■ (63) 

ii ji 

with \Piy\ > 2 (since EiX{t) = 0). Using the upper bound supj |at.„(j)| < gives 



|cum(Pi'^")(0i),...,Pi''")(0,))| 



<i^n-/^ E E 

ti^...,t(; A:i ,. . . ,/c£ 



n 



n 



n 



/in I — I /ii: 

71 



(64) 



X E n 



e{tt - iit^ - ju) 



X Y[\cnia{es\sePj] 
{Pi,...,P™}i.p.j=i 



Concavity of log/i„(-) implies that /i„(^) /i„(-^) < fe^( xhe Cauchy-Schwarz 

inequality (with squares of 0i and (^2) now leads to the upper bound 



Kn-^^H E E 

y^ti . .,t£ ki,...,k£ 



tl 



,-fci ]h: 



[tl + 1/2] 



^3(^3) • --(fiiiki) 



X E n 



£(tjr-i^)^(t--j^) 



(65) 
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^ 1/2 . N 1/2 

X ^ JJ^ |cum(£s|s G > < similar term > 

{Pi P,„}i.p.i=l J I J 



which, by using (51), is bounded by 



Kn 



E E 

^ ti ki . ..k^ 



n I \ n 



E 



1 ^ 



1 



^l,■■■,^e,Jl,■■■,]e 



£{t+ -ii) £{q -ji) J-^^ eiK - i. + iu) 



(66) 



{Pi,...,p„Ji.p.j=i 



1^=2 

1/2 |- >, 1/2 

< similar term > 



Note that the term cum (es|s G Pj) leads to the restriction that all v, ji/ € Pj are equal. 
We now sum over the remaining indices from fc2, ii, ■ ■ ■ ji, ■ ■ ■ leading, due to the 
indecomposability of the partition and the fact that < K, to the upper bound 



E E 

^ ^1 All , /C3 , . ..,ki; 



n I \ n 



n2^ , Y'^ 

^3(^3) • ■■(t>e{ke) > 



1/2 



X < similar term 



J=3 



and therefore to the result. 

In (ii), the generic constant K needs to be independent of i. Again, we have (64) (with 
K replaced by K^). Remember that the term cum {es\s £ Pj) leads to the restriction 
that all € Pj are equal. We denote this index by i'^' {j = 1, ... ,111). 

We start by considering the case where £ is even. For each fixed partition {Pi, . . . , Pm}, 
we can renumber the indices {!,...,£} in such a way that for every j S {1, . . . , m}, there 
exists at least one even 1/ S {!,...,£} and one odd ly £ {!,...,£} such that i'-'^ = i^y or 
i^-'^ = jjy. This can be derived from the indecomposability of the partition and the fact 
that \Pk \ > 2 for all k. The Cauchy-Schwarz inequality now yields as an upper bound for 
(64) 



E 

{Pl,...,Pr^}l.p.j = l 



Y[ |cum(e,|se Pj) 
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E E n 

^ti ,...,ti ki,...,ke f even 



2/[i,. + l/2] 



X E n 

X < the same term with . . . JJ^ . . 

L j odd 



1 



t{tt -iu) (-{tu -ju) 
. 1/2 



(67) 

, 1/2 



where = i^^'' if ij/ G Pj and jj^ = i^^' if G Pj . By using relation (51), we have 



< 



^ ^ £(i^-z^) £(t^ ->) " 'j^ (-{K-iu+ju) 



^E- 



that is, the first bracket in (67) is bounded by 

't 



E n 

^t^,fc^;i/ even u even 



' n ' 



[t^ + 1/2] 



1 2 



E n 

<ifV/^ n ^2':V.). 



^(1) _ _ _ i,(rn) y even 



1 



1 



t{tZ - i^) t{t-^ -iu) 



1/2 



The same apphes for the second bracket in (67). Since the number of indecomposable 
partitions is bounded by 4^(2^)!, we obtain (62). 

The case where i is odd is a bit more involved. For each partition {Pi,...,P„i} 
with m < £, the result follows as in the case of £ even. For m — £, we can renum- 
ber the indices {!,...,£} such that each contains exactly one element of {iv,jv} 
and (where i^+i =ii,j£+i = ji). For simplicity, we treat the case where 

Pu ~ {iiJjji^+i} [v = 1, • ■ • ,^) (the other cases follow analogously). We obtain, with the 
Cauchy-Schwarz inequality as an upper bound for each partition in (64), 



K^^-^Y^) E E n 

ti^ki \.t-\_,...,tt-\ ki,...,ki-i j oven 



[tj + 1/2] 



X E n 



l{tt - i^) £{tu - ii,-i) 



1/2 
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X < the same term with . . . . . . > , 

I j=l,3....J~2 ) 

where io = ie- By using relation (51), this is bounded by 



[j=2 ) tt,kt 



n 



The Cauchy-Schwarz incquahty now yields, with (51), 



0=2 



X n ^ • 



' ^ ^ 0._i(%i,-fc.-i)^^fe^(i^i^) 

.tiM,U-uke-itiM ^^^^ 



which finally leads to (ii). 



(iii) Follows from (i) since pi'^,':\^)^ < Kp^i^pf + f Poo(<^) sn^ jV {^{- , j)) ■ □ 

Proof of Theorem 2.5. We have, for each (denoted by (j) for simplicity) and fc 7^ 0, 

^ exp(-ifcA) - 1 ^ 
(/)(u, k) = / (j)R{u, dA), 







where 4>r{u, dA) is the signed measure corresponding to (j)R{u, A) := lim^jA 0i?,(u, /^) (since 
is of bounded variation in A, the limit exists; for the same reason, 4>r{u, dA) is a signed 
measure). This implies, for fc 7^ 0, 

K K 

sup I (/)(■«, fc) I < — supy(0(M, •)) = |^||(/!>|ioo,v and 
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(68) 

and, for fc = 0, 

sup|<^(w,0)| <27t||0|U,oo and F(<^(-,0)) < 27r|l</)|jv,o,. 

u 

Thus, Poo(0) is not necessarily bounded and Theorem 5.3 cannot be apphed. The trick 
now is to smooth A) in the A-direction and to prove asymptotic normaUty instead for 
the resulting sequence of approximations: Let k{x) := -^^^ exp{— ^a;^} be the Gaussian 
kernel, := fc(|) and 

/oo 
kb{X - (j){u, fj.) dn 

with 6 = 5„ ^ as n — > oo (where (/)(w, = for > tt). We have 

4>n{u,k) ^ (i>{u,k)kb{k) (69) 
with kb{k) — exp(— fc^6^/2). We obtain, from Lemma 5.7(i), 

nvar[F„(0„)-F„(^)]<:|f] £ (^1 , fc^ _ ^(^1 , fc ^ 

[exp(-fc262/2) - 1]2 



(70) 



< 



Since |1 - exp(-fc262/2)| < min{l, -^^j, this is bounded by 



|fc|<i/;) |fc|>i/& 



which implies that 



^{{Fn{(j)n) - E^^„(0„)} - {F^{<P) - ^Fr,m) 4 0. (71) 

We now derive a CLT for y/ri{Fn{(f>j,n) — 'EiFn{(l)j,n))j=i,...,k by applying Lemma 5.6(i) 
and Lemma 5.7(i). We obtain from (68) and (69) that 

suv\k\(fn{k)<K\\(l)\\^y and sup |fc|y(0„(-. A:)) < 

k k 
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Furthermore, we have 

Poo(0„) < 27t||(/)|U,oo + i^||</'||oo,y V nTexp(-fc25V2) 

k=i I''! 

< ^(11-^11 

+ log(6 )||(?!'||oo,\/) 

and, with /i„(-) /(q_;^] (•), 

P2n\4'nf' = P2,n{4>7if < P2{4>7if + -Poo{4>n) SUpV {(f„{- , k)) 

n k 



Therefore, the remainder term i?„ in Lemma 5.6(i) and the higher cumulants in 
Lemma 5.7(i) converge to zero if we choose, for example, b = —. Furthermore, 



c 



'-^"^^-^..n^fc,™) = 4'<"'^1^(^„ + Oib'/') (Z, J = 1, . . . , k). 



E 



This fohows easily by application of the Cauchy-Schwarz inequality and sup„ J {(j)jn{u. A) — 
(j)j(u, A))^ dA = 0{h) (obtained with the Parseval formula as in (70)) and j ((/)j.„(u, A))^ dA < 
K. This gives the required CLT and, with (71), also the CLT for ^(F„(0j) - 
EF„((/)j) )j=i...,fc. We obtain from (54) and (55) that 

which finally proves Theorem 2.5. □ 

Proof of Theorem 2.7 and Remark 2.8. We obtain, from Lemma 5.7(ii) for the lih 
order cumulant in the case ^ > 2, 

This implies, as on page 82 of Dahlhaus (1988), that 

The result now follows in the same way as in the proof of Lemma 2.3 in Dahlhaus (1988). 

We now prove the inequalities in Remark 2.8: We obtain, from the proof of Lemma 5.5 
and an application of the Cauchy-Schwarz inequality. 



v^|EF„(</)) - ^^+(0)1 < V^P2,„(</') 



\ t=l K\k\<n ^ 



COv(X'j^\'^2+fc/2],ri' ^[t+"l/2-fc/2],n) ri'^ 



t 



n 



2 
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\k\>n ^ 

Application of Proposition 5.4 yields that the term in the bracket is of order n~^/^, leading 
to (20). (21) and (22) follow by straightforward calculation, noting that Assumption 2.1 
implies that sup,^ ^ |/(u, A)| < oo. (24) follows from an upper bound of (54) obtained by 
using (68). □ 

Proof of Theorem 2.9. The proof uses a chaining technique as in Alexander (1984). 
Let 

Br^^\ max |Xt,„| <21ognl. (72) 

\t—l,...,n J 

Lemma 5.9 gives lim„^oo P{Bn) = 1. We will replace 4> by 



(j)l{u,X)=n (t){v,\)dv (with (/)(«, A) =0 for w<0). (73) 

J u~l/n 

The reason for doing so is that otherwise we would need the exponential inequality (19) 
to hold with /92(0) instead of P2, «(</')■ Such an inequality does not hold. Instead, we 
exploit the following property of 0* : 

P2,J« = -E/ C dA=-E/ in cb{u,X)du) dX 



t=i-^^" ^ ^ t=i 



(t-l)/n 



(74) 



<Y.\ \ <l?{u,X)dudX^pl{4>). 

i_i J -n J (t-WIn 



Define 

K('/') := ^n(0 = V^(^« - ^F^Wn)- (75) 
Since the assertion and the proof of Theorem 2.7 are for n fixed, we obtain from (19), 



P(|^:(<^)|>??)<ciexp(^-C2^-^j. (76) 

On _B„, we have, by using Lemma 5.8 and Lemma 5.9, that 

|K(0) - < MFniK) - + V^|EF„(C,) - EF„ ((/.)! 



and therefore, with (26), 



sup\E:i<P)-Enm<^- (77) 
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Thus, 



pf sup \En{cj))\ > ij, B,\ < pf sup \Ki'f>)\ > B„ 



Let a :~ H^^{^^ We obtain, for any sequence {Sj)j with a = Sq > Si > ■ ■ ■ > 0, 
where Sj+i < 6j/2 witli := -^Sj+i 

For each number (5j, choose a finite subset corresponding to the definition of the 
covering numbers N{5j, $, ^2)- In other words, the set Aj consists of the smahest possible 
number TV-,- = N{Sj P2) of midpoints of p2-balls of radius 5j such that the corresponding 
balls cover Now, telescope 

00 

= KiM + E - <l>j), (79) 
3=0 

where the are the approximating functions to (j) from Aj, that is, p2{4>~ ipj) < Sj. The 
above equality holds on i?„ because Lemma 5.8 implies that 

sup I £■;((?!)- ^j) I < 5if3n(logn)2 sup p2(?!' " 0j) 



< 5K3n{\ogn)^ Sj for all n. 



Thus, 



P( sup IK (</>)! > ^,B„ 



/ \ 00 

< P sup |K('/'o)l > J +Y.N,N,+, supP( |K(<^.+i - > ^.+1) 



= / + //. 

Hence, using exponential inequality (76), we have, by definition of a, that 

/ < ci expj H^{a) - y ^1^} = ci expj -f ^1^}- (80) 
In order to estimate //, we need a particular definition of the &j. We set 



(5j+i = sup|a:; : a; < (5j/2 ; _ff$(x) > Ht],{5j) + ^ ■ ^ 
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Since J2iti 71 > 7i 2V7TT > 21og(j + 1), we obtain for term // 



j=0 

OO 



< ^ciexp < ^ciexp{-ff$(a) - 21og(j + 1)} 

j=0 j=0 



Cl 



This implies the maximal inequality (28). To prove (29), we note that \En{4>) — 
£'„(0)| = y/niBEnicj)) ~ F{(l))\, that is, instead of (77) on S„, we obtain, with (81), (82), 
(21), (24) and (26), 

(logTj)"^ rj 

sup \E*^{(f>) - £'„(0)| < 13imax{Too.y,Ty.oo,Ty.v, Too.oo} 1=— < 77- 

</)g$ Z 
The rest of the proof is the same, that is, we also obtain (29). □ 

Proof of Theorem 2.11. To prove weak convergence of i?„, we have to show weak con- 
vergence of the finite-dimensional distributions and asymptotic cquicontinuity in proba- 
bility of En (cf. van der Vaart and Wellner (1996), Theorems 1.5.4 and 1.5.7). Conver- 
gence of the finite-dimensional distributions has been shown in Theorem 2.5. Asymptotic 
cquicontinuity means that for every e, 77 > 0, there exists a T2 > such that 



liminfPi sup \E„{(j) — ■>p)\ > i] \ < e. 

" Vp2(0,^)<r2 / 

In order to see this, we apply Theorem 2.9. For fixed 77 > 0, there exists a T2 > small 
enough such that (27) holds. To see this, notice that a as r2 — > and hence, using 
assumption (30), it follows that the integral on the right-hand side of (27) also tends to 
zero if T2 ^0. As > is fixed, (26) holds for n large enough. Hence, we obtain, with 
Bn from (72), for T2 small enough. 



lim inf P sup \En{(t> — ip)\ > rj 

" \P2(0,1/')<T2 

< hm inf p( sup \Eni<l>-ij)\>ri, P„ ) + lim P{B^) 

Vp2(</.,^)<r2 / 

<3ciexp|-| jJUe. ^ 
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Proof of Theorem 2.12. Let S > 0. The assumptions of Theorem 2.9 are fulfilled for 
rj = S^/n and n sufRciently large. For those n, we obtain, from Theorem 2.9, 



pf sup \F,-,{(j))~F{cj))\>s) =p( sup |S„(0)| > 



< 3ci exp 



C2 

' 4 



□ 



Lemma 5.8 (Properties of i^„(</)^)). Suppose Assumption 2.1 is fulfilled and 
(j)*j^{u,\) — n j^_^^_^ (f){v,X)dv (with (j)(v, X) = for V < 0). We then have, with X(^ny.= 
maxt=i,..._„ \Xt,n\, 

|F„(0)-F„(C)|<i^iX2„Y||^||v,vi^ + ll<^i|v'oo-), (81) 



|EF„(0)-EF„(</);)| <A'2||0||y,oo-, 

71 

Proof. We have, with (68), 
\F^{<I>) ^ F,M.)\ 



(82) 
(83) 



-,A -0; -,A J„ -,A dA 



ft/n 
t-l)/r 



1 

-E 

n ^ 

t=i 

^o(4„))- E ^(^(■'^)) 

A;— — n 

<ifi4„Yy^(<^)i^+supn^(.,A))i) 



-,-fc - (f>{u,-k) 
n ' 



du 



Furthermore, we obtain, with Proposition 5.4 and (68), 

1 1 '1 " pt/n 

\EF,,{<t>)-EFr,{cj^:)\<--J2 E " / 



-,-fc - (t){u,~k) 



du 



<i^2 sup F ((/)(•, A)) 

A n 



X \cOv{X[t^i/2+k/2],n,X[t+i/2-k/2],n\ 
1 



(83) has been proven in Lemma A. 3 of Dahlhaus and Polonik (2006). 



□ 
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Lemma 5.9. Suppose Assumption 2.1 is fulfilled with Elefl*^ < for all k €l>i. Let 
niaxt=i^...^„ l^t,n|- then have 

P(X(„) >21ogn) = 0(n-i). 

That is, we have P(i3Jj) = 0{n^^) for the set i?„ from (72) (used in Theorem 2.9). 

Proof. From (3), we have supj kt.nO)! < toq < oo. The monotone convergence 

theorem and Jensen's inequahty then imply 



j = -oo 



leading to 



P(X(„) > 2 logn) < n max P(Xt,„ > 2 logn) 

t—l,....n 



mo 



g2 logn 



n ' — ' k\ 

k=0 



•0. 



□ 



Appendix: Proof of Proposition 2.4 

We only give the proof for tvAR processes {q~Q). The extension to tvARMA processes 
is then straightforward. The proof is similar to that of Kiinsch (1995), who proved the 
existence of a solution of the form (2) under the assumption that the functions ai (u) are 
continuous. Let 



( -ai{u) -a2{u) 
1 





V 



-ap{u)\ 




1 / 



and cx{u) = a.{0) for u < 0. Since det(Ai?p — a.{u)) = Ap(^^^q aj(u)A -'), it follows that 
S{a.{u)) < for all u where 6{A) := max{|A| : A eigenvalue of A}. Let 



at 



t- 



11 



t-.] 



and 



3=0 
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It is easy to check that Xt^n is a solution of (12) provided the coefficients are absolutely 
summable. 

To prove this, we note (cf. Householder (1964), page 46) that for every e > and 
u S [0, 1], there exists a matrix M{u) with 



\\oi{u)\\m{u) < S{a{u))+e, 



(84) 



where ||A||a/ := sup{P2;||m : ||.t||m = 1} and = \\M-^x\\i = J2Li \{M'^x),\. Since 

the ai{u) arc functions of bounded variation (i.e., the difference of two monotonic func- 
tions), there exists for all e > a finite partition of intervals /i U • • • U /m = [0, 1] such 
that \ai{u) — ai{v)\ < e for all i whenever w, v arc in the same Ik- Let Mk '-= M{uk) for 
an arbitrary Uk G Ik- Therefore, m (and the partition) can be chosen such that 



5 

1+2 



< 1 



for all V IE Ik- 



(85) 



We now replace the first interval Ii by h U (— oo,0) (remember that <y.{u) = Q:(0) for 
u < 0). There exists a constant cq such that ||-B||i := J2i j l-^i-jl ^ co||i?||A4 for all k. For 
t and n fixed, we now define Lk '-= {£>0: ^ <E Ik} and Lk.j := ifc n {0, . . . , j — 1}. Then 



\at,nij)\ 



Vf=0 



t-j 



< 



J-1 



< 



n 

k=l 



n " 



t-e 



t-j 



k=l 



n 



t-e 



Mk 



t-J 



t-J 



< Cq" supcr(w) pl^fc jl — Kp' (since m is fixed). 



fc=i 



that is, we have proven (3). Since ||a(^)-a(^)|li = ELi we obtain, 

with similar arguments and 

a{u,j) := {a{uy)iia{u), 



E 



t=i 



n 



n j-1 

t=i fc=i 



n " 
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E 



t 

OL I — 

n 



t-k 



t=i 

n j-1 p 

^EE^op'^E 

t=l k=l i=l 

<Kfp>-\ 
that is, (5). (4) and (6) follow similarly. 



-(T| - 

n 



t 

Olr \ — 

n 



Co P 



□ 
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